Papers by H S V N S Kowndinya Renduchintala
POSIX: A Prompt Sensitivity Index For Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are sensitive to minor variations in prompts, such as spelling errors, alteration of wording or the prompt template. |
| Approach: | They propose a PrOmpt Sensitivity IndeX to measure prompt sensitivity . they use this to compare prompt sensitability of various open source LLMs . |
| Outcome: | The proposed method can measure and compare prompt sensitivity of open source LLMs. |
Heterogeneity in Formal Linguistic Competence of Language Models: Is Data the Real Bottleneck? (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit a puzzling disparity in their formal linguistic competence, even after training on trillions of tokens. |
| Approach: | They pre-train Large Language Models on 100M-token corpora and inject a minimal amount of synthetic data targeting specific linguistic phenomena into the model. |
| Outcome: | The proposed intervention significantly improves model performance in 8 out of the 9 worst-performing BLiMP paradigms. |
INGENIOUS: Using Informative Data Subsets for Efficient Pre-Training of Language Models (2023.findings-emnlp)
Copied to clipboard
H S V N S Kowndinya Renduchintala, Krishnateja Killamsetty, Sumit Bhatia, Milan Aggarwal, Ganesh Ramakrishnan, Rishabh Iyer, Balaji Krishnamurthy
| Challenge: | Pre-trained language models have a remarkable improvement in generalization capability . however, this leads to prohibitively long training times and a detrimental environmental impact . |
| Approach: | They propose to use submodular optimization to select highly informative subsets of training data to train multiple PTLMs using only fractions of data. |
| Outcome: | The proposed framework achieves 99% of the performance of fully-trained models using only fraction of training data. |
SMART: Submodular Data Mixture Strategy for Instruction Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fine tuning language models are manual or rely on intuition. |
| Approach: | They propose a method which uses a submodular function to assign importance scores to tasks and then use them to determine mixture weights. |
| Outcome: | The proposed method outperforms traditional methods such as examples proportional mixing and equal mixing. |